Skip to content

Accurate lub_constrain derivatives for large positive arguments - #3447

Open
jachymb wants to merge 10 commits into
stan-dev:developfrom
jachymb:bugfix/lub-constrain-stable-derivatives
Open

Accurate lub_constrain derivatives for large positive arguments#3447
jachymb wants to merge 10 commits into
stan-dev:developfrom
jachymb:bugfix/lub-constrain-stable-derivatives

Conversation

@jachymb

@jachymb jachymb commented Oct 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

AI use disclosure: This was developed with the help of Claude Code (Claude Opus 5.5, Claude Fable 5.1).

Summary

The reverse-mode partials of $y = l + (u - l) \sigma(x)$ (σ the inverse logit) use 1 - inv_logit(x): a relative error of 3.6e-8 at x = 20 and exactly 0 for x ≥ 37 (true ∂y/∂x 4.3e-16 at x = 37 for l = −2, u = 3). This is a first-order error, visible in optimization with jacobian = false. Both $\sigma(x)$ and $\sigma(-x)$ now come from $e = e^{-|x|}$, without cancellation and with one division:

$$ x \ge 0: \quad \sigma(x) = \frac{1}{1 + e}, \quad \sigma(-x) = e \sigma(x); \qquad x < 0: \quad \sigma(x) = \frac{e}{1 + e}, \quad \sigma(-x) = 1 - \sigma(x), $$

$$ \frac{\partial y}{\partial x} = (u - l) \sigma(x) \sigma(-x), \qquad \frac{\partial y}{\partial l} = \sigma(-x), \qquad \frac{\partial y}{\partial u} = \sigma(x). $$

For x < 0, σ(x) < 1/2, so 1 − σ(x) does not cancel. The matrix overloads keep e in the arena for the reverse pass, and the lp overloads compute the log Jacobian $\log(u - l) - |x| - 2 \log(1 + e)$ from the same e instead of a separate log1p_exp. OpenCL takes $\sigma(-x) = 1 / (1 + e^x)$ from the $e^x$ it already computes.

Forward mode goes through the prim scalar overloads. Their lb tangent had the same cancellation, their x tangent and second derivatives went through the fvar inv_logit, and abs made the second derivative of the log Jacobian 0 instead of −1/2 at x = 0. For autodiff types they now evaluate inv_logit only at a non-positive argument, $u - (u - l) \sigma(-x)$ for x ≥ 0 and $l + (u - l) \sigma(x)$ otherwise; for every type the log Jacobian branches on the sign of x instead of calling abs. double results are unchanged. This does not need #3446 (the fvar inv_logit fix): with #3446 alone, the lb tangent at large x and the curvature at x = 0 stay wrong.

Tests

References are mpmath values at 50 digits; tolerances are 1e-13 relative (1e-13 times the smallest normal double where the reference is 0). All three fail on develop.

  • New test/unit/math/rev/constraint/lub_constrain_test.cpp: all reverse-mode overloads (scalar, Matrix<var>, var_value, with and without lp) at x from −800 to 800.
  • New test/unit/math/mix/constraint/lub_constrain_fwd_test.cpp: the value, the tangents in lb, ub and x, the second derivative in x, and the first and second derivatives of the log Jacobian at 13 points from −700 to 700, in fvar<double>, fvar<fvar<double>> and fvar<var>. On develop 42 assertions fail: 40 at x ≥ 20, and the second derivative of the log Jacobian at x = 0 in both nested modes.
  • OpenCL: rev_derivatives_tails in test/unit/math/opencl/rev/constraint/lub_constrain_test.cpp.

Side Effects

With the Jacobian (sampling) lub_constrain is faster, up to 1.4 times with g++ 14 and clang 18 and 5.6 times with MinGW, except with g++ 14 for Matrix<var> with var bounds at N = 10⁴. Without it (optimization) it is up to 1.2 times slower, with g++ 14 up to 1.3 times for Matrix<var> at N = 10⁴ with x of mixed sign.

In-process against develop, -O3 -DNDEBUG, N = 10 to 10⁴: ratios 0.18–1.31 (with lp 0.18–1.12); paths over 5% slower:

path g++ 14.2 (WSL) clang 18 (WSL) MinGW g++ 15.2 (MSYS2)
Matrix<var>, no lp 1.00–1.08; 1.29–1.31 at N = 10⁴ 1.04–1.10 0.98–1.04
var_value, no lp 1.01–1.22 0.93–1.02 1.03–1.16
Matrix<var>, var bounds, lp 0.79–0.92; 1.10–1.12 at N = 10⁴ 0.82–0.90 0.45–0.59

Arena: +8 bytes per element without lp, none with lp, +8 per scalar call. Values within 2 ulp of u − l, lp sums within 55 ulp. Forward mode, double lp (one branch added) and OpenCL were not timed.

Release notes

Fixed the reverse-mode derivatives of lub_constrain, inaccurate from x ≈ 20 and zero beyond 37.

Checklist

  • Copyright holder: Jáchym Barvínek jachymb@gmail.com

    The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
    - Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
    - Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

  • the basic tests are passing

    • unit tests pass (to run, use: ./runTests.py test/unit)
    • header checks pass, (make test-headers)
    • dependencies checks pass, (make test-math-dependencies)
    • docs build, (make doxygen)
    • code passes the built in C++ standards checks (make cpplint)

  • the code is written in idiomatic C++ and changes are documented in the doxygen

  • the new changes are tested

@avehtari avehtari added the numerics Numerical issues label Oct 12, 2026
@stan-buildbot

stan-buildbot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor
Name Old Result New Result Ratio Performance change( 1 - new / old )
stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan 0.22 0.23 0.99 -1.45% slower
stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan 0.06 0.06 0.97 -3.2% slower
stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.93 -7.91% slower
stat_comp_benchmarks/benchmarks/arma/arma.stan 0.61 0.62 0.99 -0.63% slower
stat_comp_benchmarks/benchmarks/arK/arK.stan 3.18 3.2 1.0 -0.48% slower
stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan 43.14 43.08 1.0 0.14% faster
stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan 0.59 0.61 0.96 -3.96% slower
stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan 4.45 4.48 0.99 -0.83% slower
stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 21.02 22.67 0.93 -7.84% slower
stat_comp_benchmarks/benchmarks/garch/garch.stan 0.89 0.93 0.96 -4.26% slower
stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan 0.11 0.11 1.0 0.49% faster
stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan 8.3 8.31 1.0 -0.11% slower
stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan 6.34 6.56 0.97 -3.45% slower
stat_comp_benchmarks/benchmarks/sir/sir.stan 170.46 169.46 1.01 0.59% faster
performance.compilation 385.01 388.87 0.99 -1.0% slower
Mean result: 0.9785669609363813

Jenkins Console Log
Jenkins Build Stages
Commit hash: 017f091b63e5f89b01de871d2a7ead2d78127331

Machine information 
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

CPU:

Architecture:                            x86_64
CPU op-mode(s):                          32-bit, 64-bit
Byte Order:                              Little Endian
Address sizes:                           43 bits physical, 48 bits virtual
CPU(s):                                  256
On-line CPU(s) list:                     0-255
Thread(s) per core:                      2
Core(s) per socket:                      64
Socket(s):                               2
NUMA node(s):                            2
Vendor ID:                               AuthenticAMD
CPU family:                              23
Model:                                   49
Model name:                              AMD EPYC 7742 64-Core Processor
Stepping:                                0
Frequency boost:                         enabled
CPU MHz:                                 1496.698
CPU max MHz:                             3416.0681
CPU min MHz:                             1500.0000
BogoMIPS:                                4491.56
Virtualization:                          AMD-V
L1d cache:                               4 MiB
L1i cache:                               4 MiB
L2 cache:                                64 MiB
L3 cache:                                512 MiB
NUMA node0 CPU(s):                       0-63,128-191
NUMA node1 CPU(s):                       64-127,192-255
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Old microcode:             Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec rstack overflow:      Mitigation; Safe RET
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Mitigation; IBPB before exit to userspace
Flags:                                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es

G++:

g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang:

clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

@avehtari avehtari mentioned this pull request Oct 18, 2026
Open
4 tasks done
@avehtari

avehtari commented Oct 18, 2026

Copy link
Copy Markdown
Member

Looks good. I fixed two things (Claude assisted) and there are two comments about PR text

Commit 8cae6d4: no division in the matrix reverse passes

(1.0 + exp_x).inverse() is a lazy Eigen expression, so each use of σ(-x) in a reverse pass divided again. The matrix overloads now do what your scalar overloads already do. They store exp(-|x|) instead of exp(x), compute σ(x) = select(x >= 0, 1, e) / (1 + e) with one division and without the INFTY guard, and take σ(-x) = select(σ(x) >= 1/2, e σ(x), 1 - σ(x)) in the reverse pass. 1 - σ(x) is exact to rounding for σ(x) <= 1/2. The lp overloads reuse the same array: log1p_exp(-|x|) becomes log1p(exp(-|x|)), which saves one exp per element. Arena memory does not change, and scalar and matrix inputs now get the same σ values.

Time relative to your head commit (Triton, Xeon E5-2680 v3, g++ 14.3, -O3 -DNDEBUG, one core, 5 interleaved rounds, median; inputs uniform in [-6, 6]; one call is value, sum, grad and recover_memory):

path n = 10 100 1000 10000
Matrix<var>, scalar bounds 0.93 0.86 0.91 1.03
the same with lp 0.86 0.79 0.81 0.90
var_value, scalar bounds 1.12 0.87 0.82 0.83
the same with lp 0.94 0.69 0.67 0.67
Matrix<var>, var bounds, lp 0.90 0.80 0.82 0.95
Matrix<var>, vector lb, lp 0.90 0.84 0.87 0.89
Matrix<var>, vector lb and ub, lp 0.89 0.85 0.86 0.92

Relative to develop, the head commit took 1.01 to 1.17 times as long on these paths, and the new commit 0.70 to 1.14. One or two of the five rounds were up to 24 % slow on the shared node, so treat differences below about 5 % as noise. The var_value result at n = 10 (1.12) had a spread of 5 %, so it is probably real, about 30 ns per call.

Commit 456e257: forward-mode derivatives in prim/constraint/lub_constrain.hpp

Forward mode goes through the prim scalar overloads, which this PR did not change. On develop and on your head commit I measured two defects against mpmath (lb = -2, ub = 3):

  • Autodiff of (ub - lb) * inv_logit(x) + lb gives the lb tangent as 1 - σ(x): relative error 3.6e-8 at x = 20 and 1.0 from x = 37. The x tangent and the second derivative in x go through the fvar inv_logit, which has the same cancellation until Accurate derivatives of inv_logit and log_inv_logit_diff at large arguments #3446 is merged.
  • The log Jacobian uses -abs(x) - 2 log1p_exp(-abs(x)). The fwd abs returns fvar(0, 0) at exactly 0, so the second derivative there is 0 instead of -0.5, in fvar<fvar<double>> and in fvar<var>. x = 0 is where init = 0 starts.

For autodiff types the commit evaluates inv_logit only at a non-positive argument: ub - diff * inv_logit(-x) for x ≥ 0 and lb + diff * inv_logit(x) otherwise, with the branch on value_of_rec(x). That fixes the lb and x tangents and the second derivative without #3446. Arithmetic types keep diff * inv_logit(x) + lb, so double results do not change bitwise (prim/constraint/lub_transform_test compares them exactly). The log Jacobian branches on the sign in the same way instead of calling abs; for doubles that gives the same bits as before.

The new test test/unit/math/mix/constraint/lub_constrain_fwd_test.cpp checks the value, the tangents in lb, ub and x, the second derivative in x, and the first and second derivatives of the log Jacobian at 13 points from -700 to 700, in fvar<double>, fvar<fvar<double>> and fvar<var>. The references are mpmath values at 50 digits; the tolerance is 1e-13 relative, floored at the smallest normal double. With the prim header of your head commit, 42 assertions in both test cases fail: 40 at x ≥ 20 (the lb and x tangents and the second derivatives), and the second derivative of the log Jacobian at x = 0 in both nested modes. I put the test in a new file so that it does not conflict with the lub_constrain_1_test.cpp change in #3446. The two PRs change different files. With both merged into develop e4745c8, all 17 CPU suites that either PR changes or adds pass, including the forward-mode lub_constrain test of #3446.

Suggestions for the PR text and the tests:

  1. The PR text says "1e-13 relative, floored at the smallest normal double", but expect_rel in the rev test uses 1e-13 * std::fabs(expected) and the OpenCL test has no floor either. An expected value of 0 (x = ±800, and dlogJ at x = 0) therefore needs an exact 0. These assertions pass today. I would add the floor, as the new forward-mode test does, or change the sentence.

  2. The PR text says +16 bytes per element for matrices. Develop already keeps inv_logit_x in the arena and the PR adds one array, so it is 8 bytes. The forward-mode part of the PR text ("Forward mode has the same defect ... see also Accurate derivatives of inv_logit and log_inv_logit_diff at large arguments #3446") can now point to the second commit.

@stan-buildbot

stan-buildbot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor
Name Old Result New Result Ratio Performance change( 1 - new / old )
stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan 0.23 0.22 1.01 1.09% faster
stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan 0.06 0.06 0.98 -2.24% slower
stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan 6.33 6.59 0.96 -4.0% slower
stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 1.0 0.4% faster
stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan 8.29 8.26 1.0 0.44% faster
stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan 4.5 4.44 1.01 1.42% faster
stat_comp_benchmarks/benchmarks/sir/sir.stan 219.18 219.02 1.0 0.08% faster
stat_comp_benchmarks/benchmarks/garch/garch.stan 0.9 0.94 0.95 -4.96% slower
stat_comp_benchmarks/benchmarks/arma/arma.stan 0.63 0.61 1.02 2.44% faster
stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan 43.39 43.19 1.0 0.44% faster
stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan 0.59 0.6 0.98 -1.96% slower
stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 21.01 22.64 0.93 -7.72% slower
stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan 0.11 0.11 1.01 0.74% faster
stat_comp_benchmarks/benchmarks/arK/arK.stan 3.18 3.17 1.0 0.25% faster
performance.compilation 374.53 378.29 0.99 -1.01% slower
Mean result: 0.9910349884894918

Jenkins Console Log
Jenkins Build Stages
Commit hash: a2236819d562837e97a9bfe97dc9ebd6d8613729

Machine information 
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

CPU:

Architecture:                            x86_64
CPU op-mode(s):                          32-bit, 64-bit
Byte Order:                              Little Endian
Address sizes:                           43 bits physical, 48 bits virtual
CPU(s):                                  256
On-line CPU(s) list:                     0-255
Thread(s) per core:                      2
Core(s) per socket:                      64
Socket(s):                               2
NUMA node(s):                            2
Vendor ID:                               AuthenticAMD
CPU family:                              23
Model:                                   49
Model name:                              AMD EPYC 7742 64-Core Processor
Stepping:                                0
Frequency boost:                         enabled
CPU MHz:                                 1492.577
CPU max MHz:                             3416.0681
CPU min MHz:                             1500.0000
BogoMIPS:                                4491.85
Virtualization:                          AMD-V
L1d cache:                               4 MiB
L1i cache:                               4 MiB
L2 cache:                                64 MiB
L3 cache:                                512 MiB
NUMA node0 CPU(s):                       0-63,128-191
NUMA node1 CPU(s):                       64-127,192-255
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Old microcode:             Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec rstack overflow:      Mitigation; Safe RET
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Mitigation; IBPB before exit to userspace
Flags:                                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es

G++:

g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang:

clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

@jachymb
Tests: floor the lub_constrain tolerances at the smallest normal double
52828dd 
The rev and OpenCL tail tests used 1e-13 * |expected| as the tolerance,
so a zero reference (x = +-800, dlogJ/dx at x = 0) demanded an exact 0.
Use 1e-13 * max(|expected|, DBL_MIN), as the forward-mode test does.
@jachymb
Keep -|x| out of the arena in the lub_constrain lp overloads
880c7ce 
The four lp matrix overloads stored -|x| in the arena although only the
forward-pass log Jacobian reads it, so their arena use was 8 bytes per
element above develop. Compute it in that expression instead. In the
scalar-bound overload sum log1p(e) and |x| separately: with the AoS view
of x in the same expression Eigen does not vectorize log1p, which made
that path 1.3 to 1.55 times slower with MinGW.

Also drop an unused sign() evaluation in a reverse pass and state the
helper's switching condition precisely in its comment.
@stan-buildbot

stan-buildbot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor
Name Old Result New Result Ratio Performance change( 1 - new / old )
stat_comp_benchmarks/benchmarks/gp_regr/gp_regr.stan 0.23 0.23 1.0 0.26% faster
stat_comp_benchmarks/benchmarks/gp_regr/gen_gp_data.stan 0.06 0.06 0.97 -3.3% slower
stat_comp_benchmarks/benchmarks/low_dim_gauss_mix/low_dim_gauss_mix.stan 6.33 6.68 0.95 -5.41% slower
stat_comp_benchmarks/benchmarks/low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.96 -4.4% slower
stat_comp_benchmarks/benchmarks/irt_2pl/irt_2pl.stan 8.37 8.34 1.0 0.36% faster
stat_comp_benchmarks/benchmarks/gp_pois_regr/gp_pois_regr.stan 4.46 4.52 0.99 -1.13% slower
stat_comp_benchmarks/benchmarks/sir/sir.stan 219.76 221.26 0.99 -0.69% slower
stat_comp_benchmarks/benchmarks/garch/garch.stan 0.89 0.94 0.95 -5.68% slower
stat_comp_benchmarks/benchmarks/arma/arma.stan 0.62 0.61 1.01 0.74% faster
stat_comp_benchmarks/benchmarks/pkpd/one_comp_mm_elim_abs.stan 42.48 43.49 0.98 -2.37% slower
stat_comp_benchmarks/benchmarks/pkpd/sim_one_comp_mm_elim_abs.stan 0.59 0.66 0.89 -12.25% slower
stat_comp_benchmarks/benchmarks/low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 21.08 23.42 0.9 -11.12% slower
stat_comp_benchmarks/benchmarks/eight_schools/eight_schools.stan 0.11 0.11 0.98 -2.09% slower
stat_comp_benchmarks/benchmarks/arK/arK.stan 3.19 3.26 0.98 -1.98% slower
performance.compilation 365.08 378.71 0.96 -3.73% slower
Mean result: 0.9672250786127726

Jenkins Console Log
Jenkins Build Stages
Commit hash: 8774185548440f572c1900f4a344e9ba089df121

Machine information 
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

CPU:

Architecture:                            x86_64
CPU op-mode(s):                          32-bit, 64-bit
Byte Order:                              Little Endian
Address sizes:                           43 bits physical, 48 bits virtual
CPU(s):                                  256
On-line CPU(s) list:                     0-255
Thread(s) per core:                      2
Core(s) per socket:                      64
Socket(s):                               2
NUMA node(s):                            2
Vendor ID:                               AuthenticAMD
CPU family:                              23
Model:                                   49
Model name:                              AMD EPYC 7742 64-Core Processor
Stepping:                                0
Frequency boost:                         enabled
CPU MHz:                                 1496.491
CPU max MHz:                             3416.0681
CPU min MHz:                             1500.0000
BogoMIPS:                                4491.85
Virtualization:                          AMD-V
L1d cache:                               4 MiB
L1i cache:                               4 MiB
L2 cache:                                64 MiB
L3 cache:                                512 MiB
NUMA node0 CPU(s):                       0-63,128-191
NUMA node1 CPU(s):                       64-127,192-255
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Old microcode:             Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec rstack overflow:      Mitigation; Safe RET
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Mitigation; IBPB before exit to userspace
Flags:                                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es

G++:

g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang:

clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

@jachymb
Move the value_of_rec include of prim lub_constrain next to Eigen.hpp
527ab51 
At the end of the list it sat next to sum.hpp, which stan-dev#3470 removes as
unused, so the two PRs conflicted on that line. Placed after Eigen.hpp
it merges cleanly with stan-dev#3470.
@jachymb

jachymb commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks! I kept both commits and re-measured them.

  • Forward mode: confirmed.

  • Speed: the lp paths are now faster than develop: up to 1.4 times with g++ 14 and clang 18, and 5.6 times with MinGW. The exception is g++ 14.2 on my Ryzen at N = 10⁴:

    • Matrix<var> without lp takes 1.29–1.31 times as long;
    • Matrix<var> with var bounds and lp takes 1.10–1.12 times as long.

    The slowdown disappears when all x have the same sign. So it is probably the selects over the AoS views, compiled as branches, perhaps hardware-specific. Listed it in Side Effects.

  • 880c7ce: the lp overloads no longer keep -|x| in the arena, so they use develop's memory. The scalar-bound sum is split, so log1p still vectorizes for Matrix<var>.

  • I updated the PR text to match.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

numerics Numerical issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jachymb @stan-buildbot @avehtari